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Abstract 

In this paper, wc show that there exists a symmetric gossip algorithm that converges in 
■ finite time if and only if the number of network nodes is a power of two. We also show that 

c/2 ' there always exists a globally finite-time convergent gossip algorithm despite the number 

of nodes if asymmetric gossiping is allowed. In addition, an "all-or-nothing" theorem is 
, established on the finite-time convergence of distributed averaging algorithms: for a class of 

£S| ■ averaging algorithms defined by matrix sequences selected from a countable set of stochastic 

' matrices, finite-time convergence cither holds for all initial conditions, or fails for almost all 

' initial conditions. 

\o ' 
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^ ■ 1 Introduction 

Various gossip algorithms, in which information exchange is always carried out pairwise among 
the nodes, have been widely used to structure distributed computation, optimization, and signal 
processing over peer-to-peer, sensor, and social networks [21 [H [HJ [51 [TO], [Til HI El [7]. Gossip 
averaging plays a fundamental role in the study of gossip algorithms due to its simple nature 
and wide application. 

Consider a network with node set V = {1, . . . ,n}. Let the value of node i at time k be 



*This work has been supported in part by the Knut and Alice Wallenberg Foundation, the Swedish Research 

Council and KTH SRA TNG. 

'The authors are with ACCESS Linnaeus Centre, School of Electrical Engineering, Royal Institute of Tech- 
nology, Stockholm 10044, Sweden. Email: {guodongs, mikael j , kallej}@kth. se 



1 



Xi(k) € R 1 for k > 0. Let Pq, . . . , . . . be a sequence of matrices satisfying 

P k ^M = {M ij =I- {Gi ~ £j)( ^ ~ Bj)T : 1 /;}. 

where e m = (0 ... 1 . . . 0) T is the n x 1 unit vector whose m'th component is 1. Denote 
x(k) = (xi(k) . . . x n (k)) T . Then a symmetric deterministic gossip algorithm is defined by 

x(k + l) = P k x{k). (1) 

Algorithm ([1]) and its variations have been extensively studied in the literature for both 
randomized and deterministic models. Karp et al. [2] derived a general lower bound for syn- 
chronous gossiping; Kempe et al. [3] proposed a randomized gossiping algorithm on complete 
graphs and determined the order of its convergence rate. Then Boyd et al. [5] established both 
lower and upper bounds for the convergence time of synchronous and asynchronous randomized 
gossiping algorithms, and developed algorithms for optimizing parameters to obtain fast con- 
sensus. Fagnani and Zampieri discussed asymmetric gossiping in [8]. Liu et al. [9] presented 
a comprehensive analysis for the asymptotic convergence rates of deterministic averaging, and 
recently distributed gossip averaging subject to quantization constraints was studied in |12j . 
Distributed signal processing and estimation algorithms via gossiping were discussed in |1CH [TTj . 
A detailed introduction to gossip algorithms can be found in [6]. Few works have considered the 
possibility of finite-time convergence of gossip algorithms. 

We introduce the following definition for finite-time convergence. 

Definition 1 A gossip algorithm achieves finite-time convergence with respect to initial value 
x(0) = x° € R n if there exists an integer T(x°) > such thatx(T) € spanjl}. Global finite-time 
convergence is achieved if such T(x°) exists for every initial value x° 6 R n . 

The aim of this paper is to establish the fundamental limit of gossip averaging in terms of 
finite-time convergence. Consider the following simple examples. 

Example 1 Consider a network with four nodes V = {1, . . . ,4} and initial values aii(0), . . . , £4(0) 
We define a gossip algorithm {Pk} by 

P = M 12 , P 1 = M 34 , P 3 = M 13 , Pa = M 24 

following the node pair order (1,2), (3,4), (1,3), (2,4), as illustrated in Figure 1. Then all the 
nodes reach the same value X^^=i x i(0)/4 in four steps. Therefore, finite-time convergence is 
achieved. 
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Example 2 Consider instead a network with three nodes V = {1, ...,3} and initial values 
a?i(0), . . . ,2:3(0). Then if consensus cannot be reached in one step, a common value can never 
be reached in finite time via gossip averaging because there will always be two different values in 
the network, see Figure 2. Note that the initial values from which consensus can be reached in 
one step are contained in the following set: 

{ 2xi (0) = x 2 (0) + x 3 (0) } (J { 2x 2 (0) = x x (0) + x 3 (0) } |J { 2x 3 (0) = x x (0) + x 2 (0) } , 

which is of measure zero under standard Lebesgue measure in R 3 . Therefore, finite-time con- 
vergence by gossiping is impossible for three nodes for almost all initial values. 




Figure 1: Consensus can always be achieved Figure 2: Consensus can never be achieved 

in four steps for all initial values by gossip- in finite time for almost all initial values by 

ing with four nodes. gossiping with three nodes. 

We see from the examples that the possibility of finite-time convergence via gossiping in- 
deed relies on the number of nodes. We present the following main result on the finite-time 
convergence of gossip algorithms. 

Theorem 1 There exists a deterministic gossip algorithm {-Pfc}o° > Pk £ M,k > 0, that con- 
verges globally in finite time if and only if there exists an integer m > such that n = 2 m . 

In fact, as will be shown in the proof of Theorem [TJ if n = 2 m for some integer m > 0, we 
can construct a gossip algorithm which converges in (nlogn)/2 steps. The idea for constructing 
such an algorithm follows from the node pair selection strategy in Example 1. 

Recall that a finite square matrix M = [niij] S H nxn is called stochastic if rriij > for all 
i,j and J2j m ij = 1 f° r an * DP- Indeed, ([T|) is a special case of distributed averaging algorithms 
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defined by products of stochastic matrices [151 EH CL7! : 



x(k + 1) = W k x(k) 




where W k € S = {W E R nxn : W is a stochastic matrix}. Let Sq C 5 be a subset of stochastic 



matrices. We define 



.r 



€ R n : 3W , . . . , W s G 5 , s > s.t. W, ■ ■ ■ W x € span{l} 



Let M(-) represent the standard Lebesgue measure on R™. We have the following result for 
the finite-time convergence of general averaging algorithms. 

Theorem 2 (All or Nothing) Suppose Sq is a set with at most countable elements. Then either 
= ^ n or M(J% ) = 0. In fact, if ^ W 1 , then &s is a union of at most countably 
many linear spaces whose dimensions are no larger than n — 1. 

Theorem [2] implies, given countably many stochastic matrices contained in a set Sq, either 
for any initial value a; £ R n , we can select a sequence of matrices from Sq so that the obtained 
averaging algorithm converges in finite time starting from x°, or for almost all initial values, any 
averaging algorithm obtained by sequence selection from Sq fails to converge in finite time. 

For a fixed averaging algorithm, {VVfclJf with Wk 6 S, k > 0, taking Sq = {Wk,k > 0}, 
it can be immediately concluded from Theorem [2] that either this algorithm converges in finite 
time for all initial values, or fails to converge in finite time for almost all initial values. 

Noticing that A4 is a finite set and utilizing Theorems [T] and [21 we obtain the following 
conclusion. 

Theorem 3 Suppose there exists no integer m > such that n = 2 m . Then for almost all 
initial values, it is impossible to find a gossip algorithm {-Pfc}o° with P k E M,k > 0, to reach 
finite-time convergence. 

The relation between Theorem Q] and Theorem [3] is as follows. Theorem Q] indicates that if 
the number of nodes n is not a power of two, finding a gossip algorithm which converges globally 
in finite time is impossible. However, in this case, there still might exist a gossip algorithm which 
converges in finite time for some initial values, say, half of H n . Theorem [3] thus further excludes 
the possibility of the existence of such algorithms by an indeed stronger claim, which shows that 
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the initial values from which there exists a gossip algorithm converging in finite time form a 
measure zero set. 

We give some remarks on randomized algorithms. Most existing works on gossiping algo- 
rithms use randomized models El [SJ [TUl [ED CE21 CE3] • Deterministic gossiping was discussed 
in [121 E]- Although we consider deterministic algorithms in this paper, the results can still be 
easily extended to randomized gossip algorithms. Note that Theorems [H [2] and [3] do not focus on 
any particular algorithm, but on the class of algorithms defined on A4 or Sq. Therefore, even if 
some randomized structure is introduced on {-Pfc}o° and in ([I]) and (|2|), our conclusions 

still stand in the almost sure sense. Moreover, it is worth pointing out that even if So is finite 
(as .M), there will still be uncountably many different averaging algorithms in the form of 
as long as Sq contains at least two elements. 

Prom Theorem[3j we can conclude that finite-time convergence is rare for symmetric gossiping 
since in practice the number of nodes can hardly be some power to two. Now we relax the 
requirement of gossiping in way that asymmetric updates are also allowed [8]. Introduce 

M. = {l- {e '- e ^'- e ' )T : «-l,...,.}U{/-S^: ■ »}. 

Note that matrix / — ej(ej — ej) T /2 represents the case that node i updates only when it is 
gossiping with node j. 

It turns out that finite-time convergence is always possible despite the number of nodes as 
long as asymmetric gossiping is allowed. The following conclusion holds. 

Theorem 4 For any number of nodes, there always exists a deterministic gossip algorithm 
{-Pfclo 3 > Pk € M*,k > 0, which converges globally in finite time. 

2 Proofs 

We prove Theorems HJ El and [H in this section. 
2.1 Proof of Theorem CD 

(Necessity.) Suppose n = 2 ni n2 with n\ > and > 3 an odd integer. Suppose Po ; ■ ■ ■ , Pk, S 
A4 with fc* > gives an algorithm of (pQ) that converges in finite time globally. 
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Take x\, ■ ■ ■ ,x^i = and X2 n i+i, ■ ■ ■ , x n = 2 fc * +1 . Then there exists c € R such that 
Xi{k* + 1) = c,i = l,...,n. Because each element in A4 is symmetric and therefore doubly 
stochastic, average is always preserved. Thus, we have 

_ 2 fe * +1 2 ni (n 2 - 1) _ 2 fc * +1 (n 2 - 1) 

2 ni n 2 Tl2 

On the hand, it is not hard to see that c is an integer for the giving initial value since pairwise 
averaging takes place A;* + 1 times. Consequently, we have c = r 2 2 ri with < r\ < fe* + 1 an 
integer and r 2 > 1 an odd integer. 

Therefore, we conclude that 

2*.+i(na-l) 



no 

which implies 

2 fc * +1 - ri (n 2 -l) =r 2 n 2 . (3) 

This is impossible because the left-hand side of Eq. ([2D is an even number while the right- 
hand side odd. Therefore, (pQ) cannot achieve global finite-time convergence no matter how 
Pq, . . . , Pfc, . . . are chosen. 

(Sufficiency.) We need to construct a gossip algorithm which converges in finite time globally 
for n = 2 m . 

We relabel the nodes in a binary system. We use the binary number 

B!...B m , B s G {0, 1}, s = 1, . . . , m 

to mark node i if B\ . . . B m = i — 1 as a binary number. The gossip algorithm is derived from 
the following matrix selection process: 

51. Let k = l. 

52. Take 2 m_1 matrices from Ai, as the elements in the following set 

{( 34 — 6 '*)( e~ — e \^ 
I ^~~2~ ~ — : ^ n ^ e binary system, the k 'th digit of i — 1 equals 0, 



and the k 'th digit of j — 1 equals 1 j . 



In other words, we take all the node pairs (i,j), where i — 1 and j — 1 have identical 
expressions in the binary system except for the k 'th digit. Label the matrices in Vk as 



P, 



(fc_i)2m-i i • • • j -Pfc 2 m-i_i W ^ an arbitrary order. 
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S3. Let k = k + 1 and go to S2 until k = m. 



Following this matrix selection process, Pq , . . . , Pt^m-i-i gives a gossip algorithm in the 
form of (HJ. It is easy to see that the vector 

P s 2m-\_ i ' ' ' Pq X® , X G IR , S = 1, ... ,171 

has at most 2 m ~ s different elements. Thus, convergence is reached after m2 m_1 = (nlog 2 n)/2 
updates. This completes the proof. 

Remark 1 We conjecture that for n = 2' m nodes, the gossip algorithm defined by the matrix 
selection process given in the proof actually reaches the fastest convergence. In other words, 
global finite-time convergence cannot be achieved in less than (nlog 2 n)/2 steps. 

2.2 Proof of Theorem H 

Define a function 5{M) of a matrix M = [my] G E nxn by (cf. [H]) 

5(M) = max max \m a j — maj\- (4) 

Given an averaging algorithm ([2]) defined by {Wfc}^ with Wk € So,k > 0. Suppose there 
exists an initial value x° G R n for which {Wfe}o° fails to achieve finite-time convergence. Then 
obviously 5{W S ■ ■ ■ Wq) > for all s > 0. 

Claim. rank(W 8 • • • W ) > 2, s > 0. 

Let W s ■ ■ ■ Wq = (coi . . . uj n ) T with G R n . Since • • • Wo) > 0, there must be two 

rows in W s ---Wq that are not equal. Say, u\ ^ UJ2- Note that W s ---Wq is a stochastic 
matrix because any product of stochastic matrices is still a stochastic matrix. Thus, uii ^ 
for all i = 1, . . . , n. On the other hand, if u)\ = cuj2 for some scalar c, we have 1 = iojl = 
cwjl = c, which is impossible because u\ ^ 0J2- Therefore, we conclude that rank(W s • • • Wq) > 
rank(span{a;i, W2}) > 2. The claim holds. 

Suppose there exists some y G R™ such that y £ 3£s - We see from the claim that the 
dimension of ker(W s • • • Wq) is at most n — 2 for all s > and Wq, . . . , W s G Sq. 

Now for s = 0, 1, . . . , introduce 

e s = {x G R n : 3Wq, ...,W s eS , s.t. W s ■ ■ ■ Wqx G span{l}}. 
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Then Q s indicates the initial values from which convergence is reached in s + 1 steps. For any 
fixed Wq, • • • , W s € So, we define 



w a ...w 



= {zeR n : W 8 ---W z € span{l}}. 



Clearly Tw a ...w is a linear space. It is straightforward to see that @ s = \J\y a w es *^w s ...w -> 
and therefore 

oo oo 
^So = U 0S = U U ^Ws..W - 

s =o s=ow s ,...,w es 
Noticing that z € ^w a ...w implies (z — W s ■ ■ ■ Wqz) € ker(W s • • • Wo), we define a linear 
mapping 

/ : Yw e ...W ' — > ker(W s • • • Wo) x span{l} 

s.t. f(z) = {z-W s ---W z,W s ---Woz) (5) 

Suppose z\, 02 £ Yw^.-Wo with zi 7^ z<i- It is straightforward to see that either W s ■ ■ ■ Wqz\ = 
W s ■ ■ ■ W0Z2 or W s ■ ■ ■ WqZ\ ^ W s ■ ■ ■ W0Z2 implies f(z\) ^ 7(^2) ■ Hence, / is injective. There- 
fore, noting that ker(W; • • • Wo) is a linear space with dimension at most n — 2, we have 
dim(TvK s ...VKo) <n — 1, and thus M(Tvk s ...w ) = 0- Consequently, we conclude that 

M(9 s ) = m( (J T^... Wo )< Yl M(T Wa ... Wo )=0 
Wo,...,w s es w ,...,w s eSo 

because any finite power set So x • • • x So is still a countable set as long as So is countable. This 

immediately leads to 

00 00 

M(^s ) = m( y e s ) < ^M(e s ) = 0. 

Additionally, since every Q s is a union of at most countably many linear spaces, each of 
dimension no more than n — 1, %~s is also a union of countably many linear spaces with 
dimension no more than n — 1. The desired conclusion thus follows. 

2.3 Proof of Theorem H 

We first establish some auxiliary lemmas. 

Lemma 1 Let n > 1 be an integer. Then there exists a unique expression of n in the form of 

D(n) 



n = 

k=o 
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where D(n) < [(n — 1) /2j is an integer with [z\ representing the largest integer no lager than 
z, and Ok £ {0, l} for all k. 

Proof. We first prove the existence and then the uniqueness. 

(Existence.) Take k\ = max {k € Z : 2 k < n). Then n\ = n — 2 1 > is a nonnegative 
integer. If n\ = 0, the desired expression for n has been found. Otherwise, we continue to take 
&2 = max {A; € Z : 2 fc < ni} and define n2 = ni — 2 fc2 > 0. Again, the proof is ended if ri2 = 
and otherwise the process can be proceeded to define k$, . . . . Since n is a finite number, this 
process cannot last for infinity so that the existence of © for n has been proved. Moreover, it 
is straightforward to see that 

D{n) 

2(D(n) + 1) < e fc 2 * + 1 = n + 1, 

k=0 

which implies D(n) < [(n — l)/2j . 

(Uniqueness.) Suppose there exist integers D > 1 and 9k, 9k G {0, 1} such that 

D D 

^29 k 2 k = Y,0 k 2 k . (7) 

k=0 k=0 

Respectively, we define 

C* = max{/c: 9k = 1, = 0, . . . , -D}; C* = max{&: $fc = 0, A; = 1, . . . , Z)}. 

Claim. We have ("* = C*- 

We prove this claim by contradiction. Say and C* are not equal. Without loss of generality 
we assume C* > C*- According to ((7J), this leads to 

C.-i C.-1 C-i 

2^ = § k2 k - £ k2 k = Xk2 k , (8) 

k=0 k=Q fc=0 

where Xk £ {0, 1, — l}. Now we take <j* = min {/c : Xk ^ 0, k = 0, . . . , £* — l}- Then (jHJ) leads to 

C-i C-i 

2C * = E^ 2fc = 2 '*(E^ 2fc "'*)- ( 9 ) 

fe=0 fc=c* 

Noting the fact that 

C-i C-i 

£ Xk2 k ~'" = x* + £ ^ 2 "~ ? * 
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is an odd number, a contradiction aries that in ([9]), the left-hand side is even while the right-hand 
side odd. This proves the claim. 

Since = we have remove the term 2^* from both the sides of (J7]) so that 

C»-i C.-i 

0k2 k = § ^ k - (10) 

k=0 k=0 

Repeating the same argument we know that the largest powers of the nonzero items in the 
right-hand and left-hand sides of ()10p are the same. This process continues until we can verify 
that 9k = 9k for all k in ([7]). This proves the uniqueness of the expression we construct in the 
existence proof. 

The proof is complete. □ 

Lemma 2 For n nodes, there exists a deterministic gossip algorithm {-P/c}o°; Pk G A4*,k > 0, 
which converges in finite time for the initial condition X2(0) = • • • = x n (0). 

Proof. Denote a;i(0) = a and 2:2(0) = ••• = x n (0) = b. Construct a gossip algorithm as follows: 

(i) For j = 2, ... ,n — 1, node j gossips with node 1 asymmetrically, i.e., node 1 stays at a, 
and node j updates to (a + b)/2. 

(ii) Node 1 and node n do asymmetric gossiping so that they all update to (a + b)/2. 

We see all the node state converge to (a + b)/2 in n — 1 iterations. The proof is complete. □ 

We are now in a place to prove the theorem by an inductive approach. 

Denote \S\ for a set S as the number of elements in S. Based on Lemma [H we can well 
define a function as follows: 

h: 

r ^ (11) 

s.t. h(n) = \{9 ,...,9 Dn }\ with n = ££? 0*2*. 

We divide the rest of the proof into three steps. 

Step 1. For the number of nodes n satisfying h{n) = 1, it has been proved in Theorem [1] that 
there exists a globally convergent gossip algorithm using symmetric updates. 

Step 2. Suppose the theorem holds for all n satisfying h(n) = K with K > 1 an integer. This 
is to say, there exists a gossip algorithm, {-Pfc}o°, Pk £ M*,k > 0, converging in finite time 
globally if h(n) = K. 
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Step 3. Now let H(n) = K + 1. In this case, we have 

K+l 

n= ^2 dk , 4 GZ> . (12) 
fc=i 

Without loss of generality we assume d\ < d-i < ■ ■ ■ < dx+i- Then we have 

A'+l 

n = 2 dl (l + ^ 2 dfc ~ dl ). (13) 

k=2 

Now based on Step 2, for m = ^2^=2 2 dk ~ dl nodes, there exists a gossip algorithm converging 
in finite time globally in light of the fact that fr(ni) = K. Lemma [2] further ensures that 
1 + ^k=2 2 dk ~ dl nodes can converge globally in finite time. 

For n = 2 dl (l + Ylk=2 2 dfe_dl ) nodes, we can divide the nodes into m = 1 + Y^k^ 2 dk ~ dl 
groups, V\, ... , V ni , each of which has 2 dl nodes. We first make the nodes in the same group 
reach the same value in finite time, which is possible based on Theorem [TJ Next, we reorganize 
the nodes into 2 dl groups, V\, . . . , V 2 <i 1 , each of which has ri\ nodes picked from V\, ... , V ni , 
respectively. In this way, the initial values for Vi, . . . ,V 2 d 1 are just equivalent copies. Thus, 
finite-time convergence can be reached when the same algorithm (with respect to node values) 
is applied to these groups. 

Therefore, we have proved that there exists a gossip algorithm {-P/t}§°, Pk € A^*,/c > 0, 
which converges globally in finite time for all n satisfying h{n) = K + 1. 

We can finally conclude from the principle of mathematical induction that the desired con- 
clusion holds for all h{n), and thus for all n. This completes the proof. 

3 Conclusions 

In this paper, we proved that n = 2 m for some integer m is a critical finite-time convergence 
condition for symmetric gossip averaging algorithm on a network of size n. If there exists some 
integer m > such that n = 2 m , there exists a symmetric gossip algorithm reaching global finite- 
time convergence in 0{n log n) steps. Otherwise, for almost all initial conditions, finite-time 
convergence is impossible via gossip averaging. We also showed that if asymmetric gossiping 
is allowed, there always exists a globally finite-time convergent gossip algorithm despite the 
number of nodes. An "all-or-nothing" theorem was established on the finite-time convergence 
of distributed averaging algorithms indicating that for a class of averaging algorithms defined 
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by sequence selections within a countable set of stochastic matrices, finite-time convergence 
either holds for all initial conditions, or fails for almost all initial conditions. The results add to 
the fundamental understanding of distributed averaging algorithms. More challenges lie in the 
complexity of finding finite-time convergent algorithms in general, and how to construct such 
an algorithm in a distributed way. 
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